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ABSTRACT 



This paper describes an institutional research office's use 
of extensive senior survey data to examine indirectly the effectiveness of a 
career services early intervention program over a period of four years. The 
study compared responses on the senior survey by participants (N=200) in the 
sophomore career exploration seminar with responses of non-participants 
(N=4234) . Analysis indicated that seminar participants were more likely than 
other students to be female, Asian, and liberal arts majors. All career 
service items listed on the senior questionnaire were utilized more heavily 
by the career seminar participants than by non-participants. This difference 
was most notable for participation in the individual career counseling 
services, walk-in hours, job search workshops, and use of internship 
listings. Additionally, seminar participants showed a higher level of 
satisfaction with these services than did the non-participant group. 
Participants were far more likely to give the services of the career center 
an "essential" rating compared to non-participants on an importance scale. 
Participants were more likely to have completed an internship than 
non-participants and also expressed higher ratings for their overall 
satisfaction with their undergraduate experience. (DB) 
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Abstract 



Does Early Intervention Make a Difference in Career Planning? 

How an Institutional Research Office Provided Answers Without 

Implementing a New Survey 

An abundance of research exists studying the process and effects of career 
choices for college students. Most focus on student experiences in the academic 
major and the relative impact on career decisions. Few research studies examine 
the effects of career advising and planning administered through a college career 
services center, and virtually no research examines the impact of early 
intervention by a career center on students’ decision-making. 

This paper will describe an institutional research office’s use of extensive 
senior survey data to examine the effectiveness of a career services early 
intervention program over a period of four years. 



Does Early Intervention Make a Difference 
in Career Planning? 

How an Institutional Research Office Provided Answers Without Implementing a 

New Survey 

I Introduction: Career Choice and the College Experience 

An abundance of research exists studying the process and effects of career 
choices for college students (Astin, 1977, Braxton & Brier, 1989) Most of these 
studies, however, focus on student experiences in the academic major and the 
relative impact on career decisions. Few research studies examine the effects of 
career advising and planning administered through a college career services 
center, and virtually no research examines the impact of early intervention by a 
career center on students’ decision-making. 

In a related vein, studies of the undergraduate experience reveal that 
increased student involvement in college, both in academic and co-curricular 
areas, leads to greater satisfaction levels in college and greater success beyond 
college (Astin, 1984; Tinto, 1981). Little research exists, however, linking 
student satisfaction to early involvement with specific programs, particularly 
those of an applied or non-academic nature. 

This paper will describe an institutional research office’s use of extensive 
senior survey data to examine indirectly the effectiveness of a career services 
early intervention program over a period of four years. 
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II. Sophomore Career Explorations Series 

The Office of Career Services at a small research university established 
the Sophomore Career Explorations Series, which was designed as an proactive 
intervention program to reach out to students early in their tenure at college. 
Sophomore students were the target of the intervention because they are required 
to declare a major at the end of their sophomore year and also would have at least 
two years to explore all of the experiential opportunities available to them. In 
addition, a small but consistent body of evidence indicates that sophomores tend 
to exhibit significantly higher levels of mature thinking about or planning for a 
career than do freshmen. (Blann, 1985; Ware & Apprich, 1980) 

The program, presented in a seminar format, was designed to enable 
students to make more effective use of career resources and to help them feel 
more prepared to make decisions regarding their post-graduate plans. The Office 
of Career Services ran a series of career planning workshops with limited 
enrollment for sophomores at intervals throughout the academic year. The only 
information about participants that was collected by Career Services was 
participant names. 

Career Services developed a set of clearly articulated goals for the 
sophomore career explorations series. The primary objective of the workshop 
was to introduce students to the first two steps career planning process: self- 
assessment and career awareness/research. Additional objectives included 
providing information regarding the availability and access to career related 
resources and a brief introduction to job search procedures and resume writing. 
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TTT . Annual Senior Survey 



Each year prior to commencement, an extensive survey of graduating 
seniors is conducted by the Office of Institutional Research. The response rate has 
consistently averaged 95% for the past ten years.* The survey items range from 
questions about a student’s profile, such as gender, race and college major, to 
attitudinal items, such as levels of satisfaction with particular university services. 
Students are routinely queried about their satisfaction with career services and 
their level of participation in these services. 

To examine the possible impact of the sophomore career explorations 
program, responses by the sophomore seminar participants to the senior survey 
were compared to non-participants completing the senior survey. The 
coordinator of the program maintained a list of participants’ names, thus 
facilitating the our ability to track four cohorts of participants and their survey 
responses. 

rv. Linking of variables/findings 

Although each years’ cohort of participants was small compared to the 
larger, non-participant population; the numbers were sufficient to examine 
significant differences between the two groups. The size of each cohort over the 
four years is presented in Table 1. 




' The senior survey is linked to the distribution of marching order cards. Students are required to return a 
completed survey in order to obtain liis/her marching order card which is ^solutely essential to have in 
order to participate in commencement activities. 



Table 1 



Profile of Cohorts 


Cohort Year 
(Seniors) 


Participants in 
Sophomore 
Explorations Series 


Non-Participants 


1995 


54 


1044 


1996 


63 


1081 


1997 


35 


1041 


1998 


48 


1068 


Total 


200 


4234 



After linking the seminar participants’ information to the senior survey 
databases, it became clear that the four cohorts were not representative of the 
general undergraduate population. More female students and more Asian students 
participated in the program than the overall population (Table 2), and a slightly 
higher proportion of seminar participants were liberal arts majors compared to 
those in the general population (93% vs. 85%). When analyses were conducted to 
look for influences of gender and race, however, these differences were not found 
to be significant compared to the differences uncovered in the responses of the 
two groups. 

Table 2 



Gender and Race Breakdowns of Participant and Non-Participant Groups 

(Total) 




Male 


Female 


Black 


Asian 


Hisp. 


White 


Other 


Participants 


22% 


78% 


3% 


21% 


4% 


66% 


6% 


Non-participants 


48% 


52% 


4% 


13% 


3% 


71% 


9% 



Significant differences were found between participant and non- 
participants’ use of career services resources, as well as in their level of 
satisfaction with each of these services. All of the career service items listed on 



the senior survey were utilized more heavily by the sophomore career 
explorations group than the non-participant group. Most notably, participation in 
the individual career counseling services, walk-in hours, job search workshops 
and use of internships listings was much higher with the sophomore seminar 
group (Graph 1). 



Graph 1 : Participation in Career Service 
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Similarly, in almost all services, participants in the sophomore career 
explorations group showed a higher level of satisfaction with these services than 
the non-participant group. Sophomore participants were more satisfied with the 
resume writing and interview skills workshops, use of internship listings and the 
recommendation writing service (Graph 2). Participants were far more likely to 
give the overall services of the career center an "essential" rating compared to 
non-participants on an importance scale (75% vs. 58%). Non-participants were 
more satisfied with some of the recruiting services and job listings. 
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Graph 2: Satisfaction with Career Services 
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On the senior survey, students were asked to describe their plans for the 
following fall, as well as their job-finding experiences. Participants were far more 
likely to have completed an internship during their time as an undergraduate (75% 
participated in an internship versus 58% in the general population). Although 
participants were not more likely than non-participants to be employed in the fall, 
more students learned of their employment through internships if they participated 
in the sophomore seminar cohort. 

Lastly, on overall satisfaction measures with their undergraduate 
experience, sophomore participants were more likely to give the institution high 
ratings, providing greater "very satisfied" ratings (41% vs. 38%) and more likely 
to indicate that they "definitely would" attend the institution again if given the 
opportunity (42% vs. 33%). Although linking these higher satisfaction ratings to 
participation in the sophomore career exploration series would be premature, one 
could argue that this fits well with Astin's theory of involvement, which states that 
the greater participation in campus life, the more satisfied students are with their 
experiences. 

Implications of the Findings for the Career Center 

It is always useful to solicit client feedback regarding an evaluation 

product. In this particular case, we contacted the former Director of the Office of 
Career Services who was instrumental in the creation of the program and the 
extended cohort analysis and the Office of Career Services staff member who is 
responsible for conducting the program. 



Both felt that this particular cohort analysis was very useful. In addition to 
the workshop evaluations which are conducted immediately proceeding the 
session; this analysis has documented the value of the program. It has been 
utilized to demonstrate that several of the long-term goals of the program are in 
fact being achieved. Most notable is the fact that participants are more likely to 
utilize the resources of the career services center and to pursue internships. ' 

The added benefit to conducting the evaluation in tandem with an on- 
going project is that the work required by the Career Services staff was minimal - 
providing only the names of students. The former director indicated that it was 
enormously helpful to have "someone else conduct the follow-up evaluation." He 
was concerned that given his time constraints and limited research expertise, that 
he would not be able to conduct a thorough non-biased evaluation and that he 
would be unable to easily locate participants. In addition, he did not have access 
to non-participant information and having data that contrasted participants and 
non-participants made a stronger case for additional support of the program. 

The current program director indicated that Sophomore Career 
Exploration Series has become a staple in the office's programming, that there is 
solid support from the Dean of Sophomores, and the only question remains is how 
to expand the program so that more sophomores are able to participate. 

VI. Summary and Broader Implications for Use of Cohorts with the Senior 
Survey 

When the Office of Career Services approached the Office of Institutional 
Research several years ago to look for ways to evaluate the effectiveness of their 
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new sophomore intervention program, the institutional research office had in 
place years . of data gathered from the senior survey about the overall effectiveness 
of their general services. With the new program, it was important to measure its 
effects in the context of broader services offered to all students, and the senior 
survey provided the logical vehicle. In fact, the current instrument is now being 
utilized for a variety of additional assessment type analyses, including advising 
programs and specific departmental activities. 

Many of the academic and student services offered on college campuses 
could benefit from this comparative analysis if they are evaluated as part of a 
general outcomes instrument. Although some specific questions about the 
services cannot be answered using this method, a more general sense of the 
overall success of the programs may be obtained without the labor and expense of 
an additional survey instrument.^ Advising programs and tutorial services are 
obvious choices as ongoing programs that would benefit from the feedback 
gathered from regularly administered surveys. 

Many campuses now administer surveys for newly matriculated students, 
including accepted applicant surveys and the national CIRP instrument. Both of 
these surveys offer the possibility of tracking students' views as they enter an 
institution — a tertiary "pre-test" of sorts. With unique student identifiers, the 
information on these surveys can be linked to other instruments, including the 
exiting senior survey. 



■ The additional analyses associated with the evaluation of the Sophomore Careers Exploration Series 
required approximately two weeks of professional research staff time. An separate evaluation of the 
program would have required not only a greater allocation of research staff time; but also would have 
required additional expenditures of monies to complete the project.. 



For our institution, the collaboration with the Office of Career Services 
began as a way to provide data quickly and efficiently to a group already in 
progress. The implications for other campus groups and services, however, 
applying the same methodology, could be far-reaching. In an era of increasing 
demands on the limited resources of institutional research staffs, maximizing our 
databases as resources for multiple groups could be one way to manage the 
incoming data request traffic. 
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